On Pinsker's Type Inequalities and Csiszar's f-divergences. Part I: Second and Fourth-Order Inequalities

We study conditions on f under which an f -divergence Df will satisfy Df ≥ cfV 2 or Df ≥ c2,fV 2 + c4,fV 4, where V denotes variational distance and the coefficients cf , c2,f and c4,f are best possible. As a consequence, we obtain lower bounds in terms of V for many well known distance and divergence measures. For instance, let D(α)(P,Q) = [α(α−1)]−1[∫ qαp1−α dμ−1] and Iα(P,Q) = (α−1)−1 log[ ∫ pαq1−α dμ] be respectively the relative information of type (1 − α) and Rényi’s information gain of order α. We show that D(α) ≥ 12V 2 + 1 72(α+ 1)(2 − α)V 4 whenever −1 ≤ α ≤ 2, α 6= 0, 1 and that Iα = α2V 2 + 1 36α(1 + 5α − 5α2)V 4 for 0 < α < 1. Pinsker’s inequality D ≥ 1 2 V 2 and its extension D ≥ 12 V 2 + 1 36 V 4 are special cases of each one of these.